1. Field of the Invention
The present invention relates to a color image enhancement technique for a video display appliance. In particular, the present invention relates to a color image enhancement device for a video display appliance which can improve the sharpness of the color image by converting the primary color image of red (R), green (G), blue (B) into a color model of luminance (L), hue (H), saturation (S) and then utilizing the LHS components.
2. Description of the Prior Art
Generally, in order to improve the sharpness of the image displayed on the video display appliance, the luminance component, which is an intrinsic color characteristic of the color image signal of an RGB color model, is used. The luminance component represents the amount of light received by the human eye without considering the color component. However, the luminance component cannot be directly detected from the image of the RGB color model, but can be detected from the image of the LHS (Luminance, Hue, Saturation) color model converted from the RGB color model.
Before describing conventional color image enhancement techniques, the human eye's sensation of the color image in relation to the above described color models will be explained.
The human eye recognizes the color image by the reaction of the brain against the stimulation of light produced on the retina of the eye. The retina has two types of light receptive bodies, i.e., cones and rods. The rods sense a dark light and receive the entire feature of the image, but cannot recognize color. The cones mainly sense a bright light and concrete portions of the image, and has a high sensitivity on color.
As shown in FIG. 1, the cones have the characteristics of three different absorption spectra S.sub.1 (.lambda.) S.sub.2 (.lambda.), S.sub.3 (.lambda.), and the respective responses to the wavelengths of the three spectra have the maximum values in the regions of blue, green, yellow green, respectively.
Assuming that the spectrum energy distribution of a light having a color is C(.lambda.), the color sensation, which can be represented by the response to the spectrum, can be illustrated by three receptive body models for color presentation as shown in FIG. 2, and can be expressed by the following equation: ##EQU1##
According to the Thomas Young's theory, any colors can be produced by combining three primary colors. Therefore, the color matching using the three primary colors as shown in FIG. 3 can be effected, based on the theory.
Substitution for the spectrum energy distribution of the light having a color as expressed by the equation 1.1 with the color matching utilizing the three primary colors will result in the color matching that can be expressed by the following equation: ##EQU2## where, P.sub.k (.lambda.) denotes a spectrum energy distribution of the primary colors, and .beta..sub.k denotes a weighting factor given to the primary colors.
At this time, if it is defined that a.sub.i,k .varies..intg.S.sub.i (.lambda.)P.sub.k (.lambda.)d.lambda., i=1,2,3, and it is substituted for the equation 1.2, the following color matching equation can be obtained. ##EQU3##
In this connection, the tristimulus value for a certain color C can be expressed by the following equation. ##EQU4##
Here, W.sub.k represents the amount of the k-th primary color for matching the standard white color. Also, the chromaticity coordinates which represent the relative size of the tristimulus value can be expressed by the following equation: ##EQU5##
Under the assumption that t.sub.1 +t.sub.2 +t.sub.3 =1, and t.sub.1 =x, t.sub.2 =y, t.sub.3 =z, a 2-dimensional color information can be presented by means of the x-y chromaticity coordinates. This is called an x-y chromaticity graph or chromaticity coordinates of International commission on illumination (CIE). However, the CIE chromaticity coordinates have a drawback that the distance between the points representing the same color difference is not uniform. To overcome such a drawback, the tristimulus value may be linearly or non-linearly converted to be represented by a uniform chromaticity scale (UCS).
Now, the color model as described above will be explained in detail. The purpose of color modeling is for an easy processing of the colors utilizing a predetermined standard model. Particularly, in a color model determined by a three-dimensional color coordinates, any colors can be represented by a single point. Most of the color models are hardware-oriented, such as color monitors, printers, etc., or purpose oriented so as to easily process color images such as animations. As hardware models, an RGB color model for color monitors and color video cameras, a color model of cyan (C), magenta (M), yellow (Y) for color printers, and a color model of luminance (Y), inphase (I), quadrature (Q) for color television broadcasting have been used. Also, for an easy processing of the color images, a color model of luminance (Y), hue (H), saturation (S), and a similar color model of hue (H), saturation (S), intensity (I) have been used.
The characteristics of the color models as described above will be briefly explained.
First, the RGB color model can be explained by means of a color cube having three-dimensional axes for presenting the RGB colors, respectively as shown in FIG. 4. Specifically, the respective corner points of the color cube represent eight colors of red, green, blue, cyan, magenta, white, yellow, and black, and colors between black and white are represented by a gray scale.
Second, the CMY color model utilizes secondary colors such as CMY instead of the RGB primary colors. Specifically, the cyan is produced by subtracting a red light from a white light, the magenta is produced by subtracting a green light from the white light, and the yellow is produced by subtracting a blue light from the white light. In dyes, however, the mixture of yellow with magenta produces red, the mixture of yellow with cyan produces green, and the mixture of magenta with cyan produces blue. Because of this property, the CMY color model can be easily applied to the color printers. The matrices between R,G,B and C,M,Y are given by the following equation: ##EQU6##
Third, the YIQ color model utilizes the characteristics that the human eye is more sensitive to luminance than hue or saturation, and thus a relatively wide bandwidth is assigned to the luminance, while a relatively narrow bandwidth is assigned to the color. The YIQ color model is mainly used for commercial color TV broadcasting. Here, the matrices between YIQ and RGB can be expressed by the following equation: ##EQU7##
Fourth, the LHS color model poses two significant and useful characteristics. In this color model, the luminance component is separated from the color component and the hue and saturation components play an important role in the human eye's color sensation. Such characteristics will provide ideal tools to the image signal processing algorithm which is based on the human eye's color sensation.
Here, the RGB color model should be converted into the LHS color model to improve the sharpness of the color image. The luminance (L) can be defined by the following equation in the LHS color model: EQU L=0.3R+0.59G+0.11B (equation 1.8)
At this time, the primary RGB colors can be normalized to r,g,b colors by the following equation: ##EQU8##
As shown in FIG. 5A, in order to define the hue and the saturation, it is assumed that a certain color point A is placed on an R,G,B color cube, and a point P penetrates a color triangle configurated by three corner points P.sub.r, P.sub.g, P.sub.b of the color cube. The hue and the saturation can be explained by the color triangle configurated by the points P.sub.r, P.sub.g, P.sub.b as shown in FIG. 5B. The hue represents the color of the spectrum, and the saturation represent the purity of the spectrum. In FIG. 5B, the hue is represented by an angle .phi. which ranges from 0 to 2 .pi.. The saturation is determined using a straight line which connects the point P and the center W of the triangle and reaches a point P' on a side of the triangle. Specifically, the saturation with respect to the color point A is obtained by dividing a straight line WP by a straight line WP'.
Referring to FIGS. 6A and 6B, assuming that a vector (P-W) is defined by a straight line from the center point W of the triangle to the point P penetrating the triangle in order to obtain the hue of the color point A, the dot product of the vector (P-W) and a vector (P.sub.r -W) from the normalized point P.sub.r to the triangle center W can be obtained by the following equation: EQU (P-W).multidot.(P.sub.r -W)=.vertline.P-W.vertline..vertline.P.sub.r -W.vertline. cos H (equation 1.10)
Here, since the coordinates of P.sub.r, W, P are (1,0,0),(1/3,1/3,1/3), (r,g,b), respectively, the distances .vertline.P-W.vertline., .vertline.P.sub.r -W.vertline. between the vectors can be obtained by the following equation: EQU .vertline.P-W.vertline.=[(r-1/3).sup.2 +(g-1/3).sup.2 +(b-1/3).sup.2 ].vertline.P.sub.r -W.vertline.=(2/3).sup.1/2 (equation 1.11)
Since the dot product of the vectors a and b becomes a.multidot.b=a.sub.1 b.sub.1 +a.sub.2 b.sub.2 +a.sub.3 b.sub.3, the dot product (P-W).multidot.(P.sub.r -W) can be obtained by the following equation: ##EQU9##
Accordingly, the hue H can be obtain from the equations 1.10, 1.11, 1.12 as follows: ##EQU10##
However, as shown in FIG. 5B, since the hue H should be considered as three parts of RG sector (0.degree.&lt;H.ltoreq.120.degree.), GB sector (120.degree.&lt;H.ltoreq.240.degree.), and BR sector (240.degree.&lt;H.ltoreq.360.degree.), the equation 1.13 can be generalized as follows: ##EQU11##
Accordingly, the saturation S as shown in FIG. 7 is obtained by the following equations: ##EQU12##
Here, since .vertline.WT.vertline. is 1/3, and .vertline.QT.vertline. on the RG sector is b, the saturation S becomes 1-3b in accordance with the equation 1.16. The generalized saturation S can be obtained by the following equation: EQU S=1-3min(r,g,b) (equation 1.17)
If the hue H and the saturation S have been obtained as described above, the conversion process of the LHS color model to the RGB color model will be explained with reference to FIG. 8.
On the RG sector (0.degree.&lt;H.ltoreq.120.degree.), b can be obtained by the equation 1.16: EQU b=1/3(1-S) (equation 1.18)
Also, r can be obtained with reference to FIG. 8. As a result, the following equation can be introduced: ##EQU13##
From the equation 1.19, r is obtained: ##EQU14## Since .vertline.P.sub.r Q.sub.r .vertline. is three times .vertline.WQ.sub.r .vertline., r is given by ##EQU15## Substituting .vertline.WQ.sub.b .vertline.=.vertline.WP'.vertline.cos(60.degree.-H)=.vertline.WQ.sub.r .vertline. and the equation 1.15 into the equation 1.21, r is given by: ##EQU16##
Since b, r are obtained by the equations 1.18 and 1.22, g can be obtained in accordance with the equation 1.9 as follows: EQU g=1-(r+b) (equation 1.23)
On the GB sector (120.degree.&lt;H.ltoreq.240.degree.), r,g,b can also be obtained in the same manner as described above, considering the changed angle of hue: ##EQU17##
On the BR sector (240.degree.&lt;H.ltoreq.360.degree.), r,g,b can also be obtained in the same manner as above: ##EQU18##
By the method as described above, the image of the RGB color model is converted into that of the LHS color model. The luminance image is obtained in accordance with the equation 1.8, and the saturation image is obtained with reference to the equation 1.17. EQU S'(x,y)=255[1-S(x,y)] (equation 2.1)
Using the equation 2.1, a range of a negative image is provided because especially in a bright area, the negative image is easily matched by heightening the luminance image and the correlation. The hue is obtained by the equation 1.14. An original image of a baboon having an image size of 512.times.512, and gray images of luminance, hue, saturation, respectively, (not shown) were made in accordance with the equations 1.8, 1.17, 2.1, 1.14. The original image was composed of 24 bits (R: 8 bits, G: 8 bits, B: 8 bits) per pixel, and the gray image was composed of 8 bits per pixel.
Comparing the images with one another, it was observed that the saturation image had a higher frequency component than the luminance image or the hue image in a bright area. FIGS. 9A to 9C illustrate gray values of 340-th lines of the gray images of luminance and saturation, respectively. It can be confirmed that the saturation waveform has a higher frequency component than the luminance waveform by the comparison of power spectrums utilizing a periodogram described hereinafter.
The hue having an angle range of 0 to 2 .pi. is represented by gray levels ranged from 0 to 255. The waveform of the hue in FIG. 9C shows a high frequency which is almost equivalent to an impulse. This is a result of employing R as a standard hue in representing the hue angle from 0 to 2 .pi.. If the hue of an image has a value close to red, it is represented by a gray level fully ranged from 0 to 255. This results in a high frequency almost equivalent to an impulse given to the pixels having similar hues of red. This property of the hue operates as an adverse element in processing the hue. Since the hue is an intrinsic property of colors, it is desirable to retain the original hue without processing.
Here, the correlation between the luminance, hue and saturation will now be explained. The cross-correlation between the luminance, saturation, and hue in each line of the baboon as shown in FIGS. 9A to 9C can be expressed by the following equation. EQU .phi..sub.xy =E[x(n).multidot.y(n+m)] (equation 2.2)
FIG. 10 illustrates the closest correlation between the luminance and saturation, while illustrating the second closest correlation between the hue and saturation. The correlation between the hue and luminance is shown to be the lowest. FIG. 12 illustrates the respective image-correlation coefficients between the luminance and saturation, hue and saturation, and hue and luminance.
However, such data do not provide a sufficient ground to determine the order of the correlation between those three image components. It is therefore more reasonable to study the two-dimensional correlation between the luminance, saturation and hue gray images. The following equation expresses the aforementioned cross-correlation based on the images (not shown) of the baboon. ##EQU19##
The resultant diagram is illustrated in FIG. 11.
In the equation 2.3, the coefficient .rho. represents an interdependency which is defined as -1.ltoreq..rho..ltoreq.1 for clear recognition. If the coefficient .rho. has a value of 0, the three image components are in an uncorrelated state. FIG. 11 shows that the images of baboon, Garden and Closeup commonly display the closest correlation between the luminance and saturation, while displaying a relatively low correlation between the hue and luminance. This means that the correlation between the luminance and hue does not greatly affect the human eyes' perception of an image and its enhancement. Accordingly, an attempt to enhance a color image by varying the hue evidently has a limit.
A power spectrum analysis by means of a periodogram will now be explained. Power spectra of the luminance, hue and saturation can be compared by means of a periodogram. The periodogram, which is a method of predicting the power spectrum by receiving a definite number of input sampling data, can be expressed by the following equation. ##EQU20##
The equation 2.6 is a so-called Hamming Window, and I(w) in the equation 2.7 represents a power spectrum. U is a normalizing coefficient.
FIG. 12 demonstrates a power spectrum with input values of the luminance and saturation as shown in FIGS. 9A and 9B by means of the equations 2.5, 2.6 and 2.7. Referring to FIG. 12, the saturation has a higher frequency component than the luminance in the high frequency band. In other words, the saturation element has a high frequency component which is not possessed by the luminance element. This means that the saturation element may play a significant role in enhancing a color image.
However, the conventional method presented to enhance a color image utilizes only the luminance component when converting the RGB color model into the LHS color model. As illustrated in FIG. 13, the equation 1.8 is used to perform the RGB/luminance conversion in the RGB/luminance conversion section 41. Enhancement of the luminance can be obtained by using the high-frequency-emphasis filter with a convolution mask in the luminance enhancement section 42 as follows. ##EQU21##
An enhanced component of the luminance can be expressed by the following equation 2.10 in the luminance/RGB conversion section 54. ##EQU22##
Based on the resultant equation 2.10, the enhanced components of the original R(x,y), G(x,y) and B(x,y) can be expressed by the following equation 2.11. EQU R'(x,y)=K(x,y)R(x,y) EQU G'(x,y)=K(x,y)G(x,y) EQU B'(x,y)=K(x,y)B(x,y) (equation 2.11)
An original image of baboon was processed with .alpha. value of the mask of the high-frequency-emphasis filter in the equation 2.9 set to "0.1" for enhancement of the luminance. The resulting image (not shown) demonstrated an enhanced color image which is attributable to an enhancement of the luminance. Since there can be either an increase or decrease of the RGB values, however, the general hue of the image in the resulting image appeared to have decreased in comparison with the baboon image. The color image is enhanced as the .alpha. value of the high-frequency-emphasis filter increases. However, white or black outline becomes excessively thick around the boundary of images or lines because of the saturation of the varied amount of R,G,B, thereby damaging intrinsic colors of adjacent images. It is a result of separate processing of the luminance from the hue and saturation which are intrinsic properties of colors.
Another method of enhancing color images by means of S (Strickland), K (Kim) and M (McDonell) is presented in FIG. 14. This method converts the RGB color model into the LHS color model, and utilizes the saturation element and luminance element as described below.
Referring to the equation used in the color enhancement system according to S.K.M. shown in FIG. 14, the luminance element L is obtained through conversion of the RGB/luminance performed in the RGB/luminance conversion section 51 by means of the equation 1.8. The saturation element S is obtained at the RGB/saturation conversion section 52 by means of the equations 1.9 and 1.17. The negative image S' of the saturation is obtained by means of the equation 2.1. The luminance enhancement section 53 receives the respective output images from the RGB/luminance conversion section 51 and RGB/saturation conversion section 52 to obtain L' (x,y) by means of the following equations. EQU L'(x,y)=L(x,y)+k.sub.1 [L(x,y)-L(x,y)]+k.sub.2 [S'(x,y)-S(x,y)](equation 2.12) EQU L'(x,y)=L(x,y)+k.sub.3 [max {L(x,y)-L(x,y):S'(x,y)-S(x,y)}](equation 2.13)
Here, the equation 2.12 is entitled as Version 1, while the equation 2.13 is entitled as Version 2. L(x,y) represents an average region of the 3.times.3 pixel window of the luminance, while S(x,y) represents an average region of the 3.times.3 pixel window of the negative image of the saturation in the equations 2.12 and 2.13. The equation 2.12 is an expression of obtaining an enhanced luminance L'(x,y) by adding a weighting value of k.sub.1 to the high-frequency-filtered value of the luminance and a weighting value of k.sub.2 to the high-frequency-filtered value of the negative image of the saturation, which are to be added to the original luminance component L(x,y). The equation 2.13 is an expression of obtaining an enhanced luminance L'(x,y) by adding a weighting value of k.sub.3 to the greater value between the high-frequency-filtered value of the luminance and the high-frequency-filtered value of the negative image of the saturation, which is to be added to the original luminance component L(x,y). An enhanced luminance component k(x,y) can be obtained at the luminance/RGB conversion section 54 by means of the equation 2.10. An enhanced R'G'B' color image can further be obtained by means of the equation 2.11.
Images resulting from simulation experiments according to S.K.M. were produced from an original baboon image, while FIG. 16C shows a resultant image when using weighting values of k.sub.1 =l and k.sub.2 =2 in the Version 1 of the equation 2.12. The resulting image showed shows a greatly enhanced color image in comparison with the original baboon image.
However, this S,K,M method is nothing more than an incorporation of the high frequency component of the saturation into the luminance component in utilizing the saturation without utilizing an intrinsic nature of the color. This method has been proven in the experiments as having a drawback of producing a color image lacking of the hue element despite an enhancement of the entire image. The k.sub.1 =1 and k.sub.2 =2 image referred to above, and an image which was a result of processing the luminance element only, manifested an obvious defect of the hue around the monkey's eyes.
As described above, the conventional method of enhancing a color image utilizing the luminance serves to enhance the image to a certain extent. However, it carries a drawback of lacking the hue in general in comparison with the baboon image. It also poses a problem of diminishing the hue while enhancing the image in proportion to the .alpha. value. Also, the method of enhancing a color image by means of S.K.M. does not utilizes the intrinsic nature of the color but merely incorporates the high frequency component of the saturation into the luminance, thereby carrying a drawback of lacking a hue component because the S,K,M method ignores the intrinsic property of the color despite an enhancement of the color image.